Non-trivial ideas require deep understanding and thinking, solid math background, excellent engineering skills, and sometimes good luck. Trivial ideas are much easier with the aid of some short formulas. We do not encourage students to generate trivial ideas based on short formulas, but this can help beginners take the first step and gain confidence.
naive combination: e.g., combine regularizers, combine network modules
kernelize: project from low dimension to high and even infinite dimension, which is very popular before the deep learning era.
from single to multiple: create multiple-XXX learning setting, e.g., multi-view learning, multi-instance learning, multi-label learning, multi-task learning.
separate common and specific components: in multi-XXX learning setting, we can separate XXX-invariant components from XXX-specific components. e.g., for multiple domains (resp, categories), we can separate domain (resp., category)-invariant components from domain (resp., category)-specific components.
combine local information with global information: jointly use local and global information from images/videos.
from discrete to continous (e.g., using integral) or from continuous to discrete (e.g., using basis).
from coarse-grained to fine-grained: generate a coarse-grained result first and then refine the coarse-grained result to fine-grained result.
from simple structure to advanced structure: e.g., from sequence network to tree network to graph network, form vector representation to matrix representation to tensor representation.
introduce auxiliary/side information to help the original task: the auxiliary/side information is usually more accessible.
learn different weights on different components: the essence of machine learning is learning weights, e.g., assign different weights on different feature dimensions (linear classifier), on different kernels (multiple-kernel learning), on different training samples (sample reweighting).
fill in the table: analyzie the existing methods and summarize them in one table. Then, find the hole in the table and fill it with your own method.
My previous research works can be organized into the following groups:
- data-centric: how to obtain training data
- input format: e.g., color space, spatial/frequency domain
- output format: redefine the output format
- basic model: upgrade CNN to transformer or diffusion model
- essential math: extract and solve the essential math problem